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(54) Method and apparatus for coding and transmitting MPEG video over the internet 



(57) The application describes a method and appa- 
ratus for encoding a video signal, wherein in order to 
transmit an inter-frame coded video signal, such as an 
MPEG-coded video signal, over a packet-based net- 
work such as the Internet, the video signal associated 
with at least one video frame, is split (102,402) into a 
high priority partition and a low priority partition. A sys- 
tematic forward error erasure/correction (PEC) code 
(108), such as a Reed Solomon (n,k) code, is then ap- 
plied to bytes in the high priority partition. The forward 
error/erasure corrected high priority partition bytes and 
the low priority partition bytes are then combined (110) 
into n packets for transmission over the packet network 
to a receiver/decoder. Each of the n transmitted packets 
contains a combination of both high priority partition da- 
ta bytes and low priority partition information bytes. In k 



of those packets the high priority partition data bytes are 
all high priority partition information bytes and in n-k of 
those packets alt the high priority partition data byte are 
parity bytes produced by the FEC coding, fy/lore specif- 
ically, for each high priority partition byte position within 
the n packets, the forward error/erasure correction code 
is applied using one high priority partition information 
byte from the sarne byte position in each of those /c pack- 
ets to determine n-k parity bytes, which are arranged, 
one byte per packet, in the n-k packets containing high 
priority partition parity bytes. If up to n-/( packets are lost 
in transmission over the packet network to the receiver 
(500,600). then the high priority partition bytes in such 
lost packets can be recovered to applying FEC decoding 
(506) to the high partition bytes in the received packets. 
The most visually significant information is thus protect- 
ed against packet loss over the network. 
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Description 

Field of the invention 

[0001] This invention relates to the transnnission and 
reception of coded video signals over the Internet, and 
more specifically, to the transmission and reception of 
video signals that have been coded using compression 
efficient inter-frame coding techniques such as those 
used in MPEG, MPEG-2, H. 261, and H.263 standards. 

Bacl<ground of the Invention 

[0002] With the exploding popularity of the public In- 
ternet in the past several years for transporting all types 
of data, there has been much recent interest in trans- 
mitting digitally encoded real-time audio and video over 
the Internet using the Universal Datagram Protocol 
(UDP). Because UDP is an unreliable protocol, network 
packet losses will likely occur and, as a result, will ad- 
versely affect the quality of the received audio and vid- 
eo. Recovery from packet losses may be performed 
solely by the receiver, or better quality can be achieved 
by involving both the sender and the receiver in the error 
recovery process In networks that support prioritization, 
such as ATM, video quality can be improved in the pres- 
ence of packet loss by using scalable video coding (see, 
e.g., R. Aravind, M. Civanlar, A. Reibman, "Packet Loss 
Resilience of MPEG-2 Scalable Video Coding Algo- 
rithms," IEEE Transactions on Circuits and Systems for 
Video Technology, Vol. 6. No. 5, October 1996) There 
is currently, however, no widespread support for priori- 
tization on the public Internet. Overviews of proposed 
methods for error recovery for streaming of audio and 
video over the Internet, which involve both the sender 
and the receiver are disclosed by C. Perkins and O. 
Hodson in 'Options for Repair of Streaming Media," In- 
ternet Engineering Task Force Internet RFC 2354, June 
1987,and G. Carle and E. Biersack in "Survey of Error 
Recovery Techniques for IP-Based Audio-Visual Multi- 
cast Applications," IEEE Network, November/Decem- 
ber 1 997. While the general methods described in these 
overviews may be applicable to IP transmission of both 
audio and video, most of the studies published where 
specific techniques have been implemented Involve au- 
dio only. Because of its higher data rates, and propaga- 
tion of errors through inter-frame coding, it is more dif- 
ficult to maintain video qualily than audio, and audio 
techniques, therefore, cannot be directly applied to vid- 
eo signals. 

[0003] Many of the currently popular schemes for 
transmitting digital video over the Internet, such as Mo- 
tion-JPEG and wavolct-basod schemes, use intra- 
frame coding. Inter-frame coding techniques, such as 
those used in MPEG-1. MPEG-2, H.261, and H.263 
standards, are generally more compression-efficient 
than intra-frame techniques., However, the inter-frame 
standards suffer more from Internet packet loss be- 



cause errors in one frame may propagate for many 
frames. An MPEG video sequence includes intra-frame 
coded (I) frames, and inter-frame predicted coded (P), 
and bi-directional inter-frame coded (B) frames. I and P 

5 frames are used in the prediction of subsequent frames 
while B frames are not used in the prediction of subse- 
quent frames. For example, consider an MPEG video 
sequence with I frames occurring every 15 frames. In 
MPEG coding, because of inter-frame prediction, all 

10 predictive P and B frames rely upon the previous I frame. 
Thus, if an error occurs while transmitting the I frame, 
the effect persists for 15 frames, or 500 ms, which is 
quite noticeable to a viewer. The received video quality 
can be improved both through error concealment tech- 

75 niques that are applied at the decoder, and by error re- 
silience techniques that are applied at the sender 
[0004] Error resilience techniques using Forward Er- 
ror/Erasure Correction (FEC) add redundant data to a 
media stream prior to transmission, so that packet loss- 

20 es can be repaired at the receiver without requiring con- 
tact with or retransmissions from the sender. Forward 
Error/Erasure Correction techniques are well suited to 
multicast applications, because they avoid the use of re- 
transmissions. The same redundant data can be used 

2B to repair the loss of different packets at separate receiv- 
ers in a multicast group. If re-transmission were used 
instead, multiple re-transmtsston requests would have 
to be sent. Forward Error/Erasure Correction tech- 
niques for multimedia generally fall into one of two cat- 

30 egories. media-independent FEC and media-specific 
FEC ( see. e.g.. C. Perkins and O. Hodson. "Options for 
Repair of Streaming Media." Internet Engineering Task 
Force Internet RFC 2354, June 1 998). 
[0005] In media-independent FEC, well-known infor- 
ms mation theory techniques for protecting any type of data 
are used. In, "Media-independent Error Correction us- 
ing RTP," Internet Engineering Task Force Internet 
Draft, May 1997 by D. Budge. R. McKenzie, W. Mills, 
and P. Long, several variations of exclusive-OR (XOR) 

40 operations are used to create parity packets from two 
or more data packets. More complex techniques such 
as Reed Solomon (RS) coding can also be used (see, 
e.g., G. Carle and E. Biersack, "Survey of Error Recov- 
ery Techniques for IP-Based Audio-Visual Multicast Ap- 

45 plications," IEEE Network. November/December 1 997). 
Reed-Solomon encoding is an example of a systematic 
forward error/erasure correction code. A systematic for- 
ward error/erasure correction code is one in which the 
information bytes are transmitted in the codeword with- 

50 out modification. Thus, in the absence of channel errors, 
no Reed-Solomon decoding is necessary to recover the 
information bytes. When an F{S(n,k) codeword is con- 
structed from byte data. /? parity bytes arc created from 
k information bytes, and a\\ n = k + h bytes are transmit- 

55 ted. Such a Reed Solomon decoder can correct up to 
any h/2 byte errors, or any h byte erasures, where an 
erasure is defined as an error in a known position. When 
RS coding is applied to protect packetized data against 
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packet loss, Ac information packets of length ) bytes are 
coded using y'RS codewords. For each RS codeword, k 
information bytes are taken from /cditferent packets (one 
from each packet), and the h constructed parity bytes 
are placed into h separate parity packets, and a\\ n = k 
+ h packets are transmitted. Because the transmitted 
packets are numbered, and packets are assumed to be 
received perfectly or not at all, the receiver can deter- 
mine which packets are missing, and thus a packet loss 
can be considered to be an erasure. Hence, if any h (or 
fewer) of the n transmitted packets are lost, the original 
k information packets can be recovered perfectly. 
[0006] A key advantage of RS coding is its ability to 
protect against several consecutive errors, depending 
on the parameter choices. The overhead rate for RS 
coding is h/k, and it is most efficient for protection 
against burst errors for targe values of k. For example, 
an RS(6.4) code and an RS(4,2) code both can protect 
against a burst length of 2 errors. But the RS(4,2) code 
has 100% overhead, while the RS(6,4) code has only 
50% overhead. Reducing the overhead percentage by 
increasing the block length, however, leads to delay be- 
cause large block lengths require buffering of large 
amounts of data prior to transmission. 
[0007] In mcdia-spocific PEG coding unlike in media- 
independent FEC coding where the multimedia stream 
is just treated as data, knowledge of the specific type of 
multimedia stream to be transmitted is used. In "Simu- 
lation of FEC-Based Error Control for Packet Audio on 
the Internet," INFOCOM, March 1998. San Francisco. 
CA by M. Podolsky, C. Romer. and S. McCanne. and in 
"Reliable Audio for Use over the Internet." Proc. INET 
'95, Honolulu, HI, pp. 171-178, June 1995. by V Hard- 
man, M A. Sasse, M Handley, and A. Watson, a redun- 
dant low-bit rate audio stream is transmitted along with 
the standard audio stream, but delayed by one packet. 
If a standard audio packet is lost, the receiver uses the 
low-bit rate version of that audio instead, received in the 
next packet. This method protects against single packet 
losses. 

[0008] In the aforenoted article by Perkins and Hod- 
son, a suggestion is made to combine media-specific 
and media-independent techniques by applying the me- 
dia-independent FEC techniques to the most significant 
bytes of a coder's output, rather than applying FEC over 
the entire multimedia bitstream. No specific information 
about how this can be accomplished is given however. 
A method for adding resilient information to inter-frame 
coded video, such as fy/lPEG video, in order to protect 
video quality against packet loss, but which has low 
overhead and low delay is desirable. 

Summary of the Invention 

[0009] In accordance with the present invention, an 
inter-frame coded video signal, such as an MPEG video 
signal, employs a data splitting function to split such a 
video stream into a high priority and a low priority parti- 



tion. Systematic Forward Error/Erasure Correction cod- 
ing is then performed on only the data in the high priority 
partition. The Forward Error/Erasure Corrected high pri- 
ority partition data and the non -Forward Error/Erasure 

5 Corrected low priority partition data are then combined 
into packets and transmitted over the same network to 
a receiver, where they are decoded. Depending on the 
degree of protection against errors or erasures offered 
by the particular FEC code that is used, the loss of one 

10 or more packets containing high priority data can be cor- 
rected with no loss of data in the high priority partition. 
The effect of the loss of the low partition data in the lost 
packet or packets, which low partition is not protected, 
has much less of a deleterious effect on the quality of 

15 the decoded video signal than would the loss of data 
from the high priority partition data. Advantageously, by 
limiting the application of the Forward Error/Erasure 
Correction to only the higher priority partition data, and 
thus protecting against loss only that "more important 

20 data", the overhead requirement is reduced for protec- 
tion against a given packet loss. 

[0010] In the preferred embodiment, a Reed Solomon 
encoder is applied to the high priority data for an entire 
frame. For each RS(n,/f) codeword, one information. 

25 byte is taken from each of k packets and the constructed 
parity bytes are placed in different packets, where n = 
k+ h. Each individual frame's data is arranged in the n 
equal length packets that contain a combination of: 
packet headers; high priority data comprising one of ei- 

30 ther information bytes or parity bytes; and low priority 
data bytes, the latter comprising only information bytes 
since no error-correction codjng is performed on the low 
priority data. The same number of bytes of high priority 
data (informatiori or parity in any one packet) are placed 

35 in each of the n equal length packets, and the same^ 
number of bytes of low priority data (information only) 
are placed in these same n packets, which together rep- 
resent the video frame. Amongst these n equal length 
packets, /c packets only contain high priority partition tn- 

40 formation bytes and h packets only contain the-high pri- 
ority parity bytes. The parity byte in each high priority 
byte position in each of theses /7 packets is formed from 
the f{S(n,k) code as it is applied to the k high priority 
partition information bytes in a corresponding byte po- 

45 sition in the k other high priority partition information- 
containing packets associated with the frame. Advanta- 
geously, arranging the packets in this manner minimizes 
the amount of overhead and delay for a given packet 
loss protection rate 

50 [0011] A receiving decoder, upon receiving the pack- 
ets associated with a frame separates the high priority 
partition bytes and low priority partition bytes in each 
packet according to the numbers of such bytes or each 
type included within each packets, which numbers are 

55 transmitted in the packet headers. BS(n,k) decoding is 
applied byte position-by-byte position across the high 
priority partition portion within the received packets. If 
up to hoi the n frame packets are lost, the RS decoding 
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process recovers each high priority byte in the lost pack- 
et or packets. Full reconstruction of the high priority par- 
tition information bytes that were transmitted in the k 
packets of the n packets that contained high priority par- 
tition data is thus effected. Although the low priority par- 
tition data in the lost packets is unrecoverable, the fully 
recovered high priority partition data enables the video 
picture to be decoded, albeit in what might be at a re- 
duced quality level for that frame or that portion of a 
franDe in which only the high priority partition information 
is available. 

Brief Description of the Drawing 
[0012] 

FIG. 1 is a block diagram of first embodiment of a 
video encoder in accordance with the present in- 
vention which uses a data partitioning MPEG en- 
coder to code and split the coded video signal into 
HP and LP partitions; 

FIG. 2 the arrangement of HP data and parity infor- 
mation, and LP data within the packets associated 
with a frame for an example of RS(4,3) encoding; 
FIG. 3 is a flow chart that details the steps for de- 
termining the nand parameters associated with a 
frame as a function of the number of LP and HP 
bytes in the frame; 

FIG 4 is a block diagram of a second embodiment 
of a video encoder in accordance with the present 
invention in which a standard MPEG encoder is 
used to encode the video signal and a data splitter 
divides the coded signal into HP and LP partitions; 
FIG. 5 is a block diagram of an embodiment of a 
video decoder in accordance with the present in- 
vention in which lost packets are reconstructed, HP 
and LP partitions reformed, and a data partitioning 
MPEG decoder is used to decode the video signal 
from the HP and LP partitions; and 
FIG. 6 is a block diagram of a second embodiment 
of a video decoder in accordance with the present 
invention in which a data merger combines the HP 
and LP partitions, which are then inputted to a 
standard MPEG decoder to decode the video sig- 
nal. 

Detailed Description 

[0013] The MPEG-2 standard (ITU-T Recommenda- 
tion H.262, "GENERIC CODING OF MOVING PIC- 
TURES AND ASSOCIATED AUDIO INFORMATION", 
July 1 995 ) includes several means for performing scal- 
able video coding, including spatial scalability, SNR 
scalability, and data partitioning. In scalable video cod- 
ing, receivers with different available bandwidths can re- 
ceive and decode appropriate representations of coded 
video, by receiving a base layer, and if bandwidth is 
available, receiving one or more enhancement layers. 



6 

The more layers received, the higher the quality of the 
decoded video. In the aforenoted-paper by Aravind, Civ- 
aniar and Reibman, data partitioning was applied to pro- 
vide protection against packet loss for transmitting an 

5 MPEG-coded video signal in a network that supports pri- 
oritization such as ATM. Specifically, protection against 
packet loss was shown to be achievable by transmitting 
the base layer at a higher priority than the enhancement 
layer(s). Since the public Internet does not currently 

TO support prioritization of packets this technique cannot 
be applied to the transmission of coded video over the 
Internet. 

[0014] The inventor has recognized, however, that an 
advantage can be achieved by using, in a non-prioritized 

75 network, such as the public Internet, a data splitting 
functionality that is used in the prior art for packet pro- 
tection over a network that does support prioritization. 
By incorporating a data splitting functionality together 
with a Forward Error/Erasure Correction functionality for 

20 transmitting an inter-frame compression-coded video 
signal over a non-prioritized public Internet, the amount 
of overhead needed for packet protection is reduced, 
while achieving the improvement in the subjective video 
quality that packet protection affords. Further, by com- 

2S bining high priority and low priority data in the same 
packets, the delay for equal overhead and protection is 
advantageously reduced. 

[0015] FIG. 1 is a first embodiment of an encoder in 
accordance with the present invention. In this embodi- 

30 ment of the present invention, an encoder which is com- 
pliant with MPEG-standardized data partitioning is used 
to split an incoming video data stream into a high priority 
partition and a tow priority partition. With reference to 
FIG. 1 , an incoming video data stream on 1 01 is inputted 

35 to a such a compliant data partitioning MPEG-standard- 
ized encoder 102 which, in accordance with the stand- 
ard, such as the MPEG-2 data partitioning standard, 
compression-codes the input video bitstream and splits 
the compression-coded bitstream into two output layers. 

40 The first layer is the base layer, referred to herein as the 
high priority (HP) partition. The second layer is the en- 
hancement layer, referred to herein as the low priority 
(LP) partition. 

[0016] As is well known to one skilled in the art of 
-^5 MPEG coding, an MPEG-coded video bitstream in- 
cludes headers at the sequence level, at the group-of- 
picture (GOP) level, at the picture (frame) level, and at 
the slice level. As is well known, a slice consists of a 
group of adjacent macroblocks, where each macroblock 
50 in itself consists of a group of four adjacent luminance 
blocks of data and two chrominance blocks. At the pic- 
ture level, a frame is classified as being as an intra- 
frame coded (I) frame, an inter-frame coded predictive 
(P) frame, or a bi-directional inter-frame predictive (B) 
5B frame. At the macroblock level, for the predictive-type P 
and B type frames, information is included which indi- 
cates whether the macroblock is inter or intra coded with 
respect to another frame, as well as motion vector infor- 
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mation. The information at the block level includes low 
frequency and high frequency discrete cosine transfor- 
mation (OCT) coonicients derived from actual pixel in- 
formation 

[001 7] In accordance with (vIPEG standards, data par- 
titioning can be effected at a plurality of different priority 
breakpoints such as above the macroblock layer, at the 
macroblock layer or within the macroblock layer Gen- 
erally, the more critical parts of the coded bitstream, 
such as the headers, the motion vectors and the low fre- 
quency DCT cocHtcients, are allocated to the HP parti- 
tion, and the toss critical data, such as the higher fre- 
quency DCT cooMtcicnts. are allocated to the LP parti- 
tion. In accordance with the standard, a priority break- 
point can be chosen for each slice, which determines 
which codeword types are arranged into which partition. 
[0018] In the embodiment ot the invention in FIG. 1, 
which uses the standardized data-partitioning MPEG 
encoder 102. priority breakpoints are determined in ac- 
cordance with the type of frame (I, R or B) in the data 
stream that is being partitioned. Specifically, in this em- 
bodiment, for each I frame, all the data is placed in the 
HP partition, with no data being placed in the LP parti- 
tion. Other embodiments may divide the data from each 
I frame between the HP and LP partitions. For each B 
frame, as much frame data as the MPEG standards re- 
lating to data partitioning allows is placed in the LP par- 
tition, with the remaining data placed in the HP partition. 
For each P frame frame data is divided between the HP 
and LP partitions so that data elements through the first 
two DCT coefficients of each block are placed in the HP 
partition, with the higher order DCT coefficients placed 
in the LP partition. Different priority breakpoints in the 
three frame types between the LP and HP partitions oth- 
er than described above for this embodiment could 
equally be used. It would be expected, however, that a 
higher breakpoint would be used for I frames than for P 
frames, which in turn, would be higher than the break- 
point for B frames, in this standards-compliant embod- 
iment, sequence, GOP, and picture headers are copied 
into both partitions for error resilience. 
[0019] In accordance with the invention, the HP par- 
tition is encoded with a systematic Forward Error/Eras- 
ure Correction code Specifically, in this preferred em- 
bodiment, Reed Solomon coding is applied to the HP 
data for a single frame across the entire frame. For this 
embodiment, only data from a single frame is operated 
upon by a Forward Error/Erasure Correction encoder/ 
packetizer 103 at a time to minimize delay that would 
be incurred if data from multiple frames needed to be 
accumulated before being coded and transmitted. In 
other embodiments, B frames as well as their immedi- 
ately following anchor frame (I or P) may be operated 
upon together by the Forward Error/Erasure Correction 
encoder/packetizer 103. 

[0020] Reed Solomon encoding is applied to the HP 
data output of the MPEG encoder 102 for the frame. As 
will be described in detail hereinafter the number of 



•bytes in the HP partition and the number of bytes in the 
LP partition are used together and separately, together 
with a maximum allowable packet length and a desired 
degree of packet protection, to determine the number of 

5 equal-length packets, n, into which, for that frame, the 
HP information bytes, the LP information bytes, and the 
protective HP parity bytes will fit. In accordance with the 
invention, each of the n equal-length packets contains 
a combination of both LP information bytes, and one of 

10 HP information and parity bytes. The parity bytes are 
determined by forming an RS(n. Ac; codeword by taking 
one HP information byte from each of /cpackets that con- 
tain the HP information bytes and placing the h parity 
bytes constructed from these /c bytes, one byte at a time. 

15 into /? different packets, where n = k+ h. Thus, the parity 
bytes in those h packets in byte position m (where m 
varies between 1 and the number of HP information 
bytes in a packet) are derived from the HP information 
bytes in that same byte position m in the k packets. 

20 Since the parity bytes in each byte position in those h 
packets are calculated from the HP information bytes in 
the same byte positions in the other k packets, each 
equal-length packet contains an equal number of HP da- 
ta bytes, the latter being either all information bytes or 

25 all parity bytes. It follows, therefore, that each equal- 
length packet then contains an equal number of LP in- 
formation bytes. Because the number of HP information 
bytes in the frame is not necessarily divisible by the 
number of /cpackets, padding bytes are added, as nec- 

30 essary, at the end of the f^^ packet. Similarly, padding 
bytes may be applied to the rf^ packet for LP information 
bytes. 

[0021] FIG. 2 shows an example of an arrangement 
of a frame group of packets for an RS (4. 3) code. As 

35 can be noted, three packets (/c= 3) contain HP informa- 
tion bytes information bytes and LP information Bytes. 
One packet contains the HP parity bytes and the 

LP information bytes. Arranging the packets in the man- 
ner illustrated by this figure minimizes the amount of 

40 overhead for a given packet loss protection rate without 
adding to delay. The packet header in each packet con- 
tains the packet number, the number of the packets in 
the frame group of packets (n), the number of packets 
with pahty data in the frame group of packets (h), the 

45 temporal reference value for the frame, the frame type 
(I. P, or B), the HP/LP priority breakpoints, and the 
number of HP and LP bytes in each packet. 
[0022] With reference back to FIG. 1 , the output of the 
data partitioning MPEG encoder 102 consists, for an in- 

50 put stream of video data associated with a frame, of two 
data streams, an H P partition data stream on output 1 04 
and a LP partition data stream on output 105, both of 
which are inputted to the FEC Codor/Packetizor 103. 
The number of bytes in the HP partition of the input 

55 frame, HP#, and the number of bytes in the LP partition 
of the input frame, LP#, are inputted to a processor 106 
within coder/packetizer 103. As will be described in de- 
tail hereinafter processor 106, using the values HP# 
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and LP#, together with a predetermined maximum 
number of bytes per frame, determines the number of 
packets, a needed to transmit the entire frame, includ- 
ing packet headers, the HP partition information bytes, 
the HP partition parity bytes, and the LP partition infor- 
mation bytes. Further, processor 106 determines the 
number of packets, h (equal to n-k), which are needed 
to protect the HP partition information bytes in the other 
k packets tor a desired level of protection. Processor 
106 also determines the byte-length of each of the n 
packets Processor 106 then determines the numbers 
of HP data or parity bytes and LP data bytes to be ap- 
portioned into each of these n packets, with LP informa- 
tion bytes allocated to each of the n packets. HP infor- 
mation bytes being allocated to the /c packets, and HP 
parity bytes being allocated to the other h packets, as 
per the example shown in FIG. 2. 
[0023] Once processor 106 has determined all the pa- 
rameters that determine the configuration of each pack- 
el and the number of such packets, a Reed Solomon 
RS(n,/() encoding is effected, byte by byte on the bytes 
within the HP partition. Specifically, the HP partition vid- 
eo stream is inputted to a byte interleaver 1 07, which in 
response to the parameters determined by processor 
106, forms sequential codewords that arc inputted to a 
Reed Solomon encoder 1 0B. For example, if the number 
of HP bytes per packet are calculated to be 650, the 1®S 
651 s^ 1301®*, et seq., bytes in the HP stream are inter- 
leaved by interleaver 107 to form an input word to RS 
encoder 1 08 in order to determine the bytes for the first 
parity byte position for the Vpackets which are to contain 
HP parity bytes. The 2"*^, 652"^, 1302"^, et seq., bytes 
are then interleaved to form the next input word to RS 
encoder 1 08 to determine the parity bytes for the second 
parity byte position for the Aj HP parity packets. In such 
a manner, the RS encoder 1 08 is sequentially presented 
with a series of /c-byte input words, each byte in each k- 
byte input word comprising one information byte from 
what will be, when re-interleaved, k different packets. 
For each /f-byte input word, h parity bytes are deter- 
mined. Thus, for each /c-byte input word, RS encoder 
108 outputs an n-byte word containing k information 
bytes and h parity bytes. 

[0024] In order to reassemble these vertically onented 
codewords, each containing bytes properly belonging in 
n different packets, back into a row-oriented packet for- 
mat, a de-interleaver 109 sequentially places each byte 
from each rvbyte output word into a separate one of n 
different rows. The first k of such n rows thus contain all 
HP information bytes and the last /lof such rows contain 
all HP parity bytes. When the n rows are reconstructed, 
each such row is combined by a packetizer 110 with a 
packet header and the calculated fixed number of LP 
partition information bytes apportioned to each packet 
from the LP output 105 of the data partitioning MPEG 
encoder 102. Each packet, then in a format as shown 
in FIG. 2 for the illustrative example for an RS(4,3) code, 
is sequentially outputted on output 111 for transmission 



onto a UDP/iP network. 

[0025] For the example of FIG. 2. if none of the pack- 
ets containing the frame information are lost in transmis- 
sion, the decoded video will be perfectly decoded (i.e., 

5 identical to that encoded). If one of the packets (or up 
to h packets in the general case for f{(n,k) coding) is 
lost, then all of the HP information in that packet is re- 
coverable using a Reed Solomon decoder The LP in- 
formation data in the lost packet could not. however, be 

TO recovered since it is not protected. The portions of the 
picture that correspond to the macroblocks whose LP 
data was received will, however, be perfectly decoded, 
and those that correspond to macroblocks whose LP da- 
ta is lost will decode only the HP data for those macrob- 
locks. Those macroblocks which are decoded using on- 
ly HP data may be noticeable to a viewer, but are unlikely 
to be visually offensive as long as the HP/LP priority 
breakpoint is properly chosen. The exact visual quality 
of macroblocks decoded using only HP data depends 

20 on this HP/LP priority breakpoint, as well as the charac- 
teristics of the video source material. I n the embodiment 
of FIG. 1 in which a standardized data partitioning 
MPEG encoder 102 is used to com press ion -code the 
input video stream and form the HP and LP partitions, 

25 the lowest level hoador copied into both partitions is the 
picture header. Thus, once any packet is lost, and with 
it the LP partition information contained therein, which 
cannot be recovered, the LP partition data that is re- 
ceived in the next received, packets cannot be properly 

30 incorporated into the decoding process since there is no 
identifiable spatial entry point with which that received 
data can be associated until a next picture header is re- 
ceived. Thus, with the embodiment of FIG. 1, the per- 
ceptual effect of lost LP partition information will extend 

35 until a packet is received that contains a next picture 
header within the received LP partition information. 
Once that picture header is received, the LP partition 
data that follows can be properly incorporated with the 
received HP partition data. 

40 [0026] As previously noted, the exact arrangement of 
the frame data into packets is a function of the number 
of bytes of HP partition information and LP partition in- 
formation in the frame, the parameters of the f{S(n,k) 
code, and the maximum packet size. For Internet Pro- 

45 tocol (IP) transmission, the maximum packet size is set, 
for this embodiment, to the Ethernet Maximum Trans- 
port Unit (MTU) size of 1500 bytes. The choice of the 
RS(n,k) code parameters depends on the number of 
packets needed to transmit the frame, on the network 

50 toss conditions, and the overhead percentage consid- 
ered acceptable. Because Internet packet loss has a 
tendency to be somewhat bursty, the k/n ratio can be 
chosen to bo lower for lower values of n and higher for 
higher values of n. Table 1 is an example of pairs of n 

55 and k values in which, based on the bursty nature of 
packet loss, the ratio k/n is set lower for lower values of 
n and higher for higher values of n. 
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Table 1 
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4 
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5 
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10 
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11 
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12 
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13 


9 


14 


10 


15 


11 



[0027] Given a list of (n,k) values such as that given 
in Table 1 , the particular (ak) pair to use tor each franne 
can be deternnined using an iterative process. FIG. 3 is 
a flowchart that details the steps that processor 1 06 fol- 
lows for determining the these parameters. After the da- 
ta partitioning MPEG Encoder 102 splits the bitstream 
into the HP partition and the LP partition, there are HP# 
bytes in the HP partition and LP# bytes in the LP parti- 
tion. At step 301, an initial estimate is formed for the 
number of packets, a needed to transmit this informa- 
tion, assuming a maximum packet size PS^g^, which in 
this embodiment is equal lo the Ethernet MTU size mi- 
nus the number of packet header bytes used. This initial 
estimate of n is formed from n - ceil {HBff + LBff)/PS^^x, 
where ceil {) is the ceiling function. This initial estimate 
of n is calculated without consideration of the HP parity 
bytes, which must bo incorporated into the frame. The 
value of /cthat corresponds to the current estimate of n 
is retrieved, at step 302, using Table 1. At step 303. the 
necessary packet size for the HBM information bytes is 
calculated from CurPacketSi7e = ceil (HB^/k) + ceil 
{LB^/n), because the HB^ information bytes are split 
among k packets (with parity bytes in the remaining h 
packets) and the LS# information bytes are split among 
n packets. At step 304, CurPacketSize is compared with 
P^max- CurPacketSize is smaller than PS^^^, the cur- 
rent {n,k) parameters may be used. If not, the packet 
size is too large, and nis incremented at step 305. Table 
1 may yield a new ^ for this incremented value of n. Cur- 
PacketSize is then recalculated using these new param- 
eters. The process ends at step 306 when the resultant 
parameters yields a CurPacketSize oi less than PS^^^^. 
The actual number, HBP, of HP bytes (information or 



12 

parity) in each of the n frames is then equal to ceil (HS#/ 
k) and the actual number. LPB, of LP information bytes 
in each of the n frames is equal to ceil (LP^/n). the total 
number of bytes, less the length of the packet header. 

5 in each frame, TPS, thus being equal to HBP+LPB. 
[0028] As an illustrative example, a P frame will be 
assumed to have a size of 5632 bytes, in which HB# = 
2129 HP bytes and Le# = 3503 LP bytes. PS^^j, is as- 
sumed to be equal to 1475 bytes, derived from a 1500 

10 byte MTU size minus 25 bytes for a packet header. The 
initial estimate of the number of packets is n=ceil 
((21 29+3503)/1 475) = ceil (3.81 8) = 4. From Table 1 . for 
n=4, it Is seen that k~3. Using these parameters, HPB, 
LPB and TPS axe calculated: HPfeceil (2129/3)=710; 

15 LP&ceil (3503/4)^876: and TPB= 710+876=1586. 
which is greater than PS^^^ of 1475. Thus, that set of 
parameters is not valid and another Iteration is neces- 
sary For this next iteration, nis incremented to 5. From 
Table 1. for n=5, it is seen that k-A. Using these param- 

20 eters. HPB, LPB and TPS are calculated. HP8=ceil 
(2l29/4)r:533; LPe=ceil(3503/5)=701; and rPe=533+ 
701=1234, which is less than PS^^^ of 1475. 
[0029] Once the parameters of n, K HPBand LBP are 
determined for the frame, the aforedescribed Reed 

25 Solomon coding and arrangement of the HP information 
and parity bytes, and LP information bytes into packets 
can be effected. Specifically, once processor 106 deter- 
mines these parameters from the partitioned HP and LP 
data for each frame, that frame's data is interleaved. RS 

30 encoded, de-interleaved and packetized for transmis- 
sion over a UDP/IP network, as previously described. 
[0030] In the embodiment of FIG. 1 , a data partitioning 
MPEG encoder 102 was used to split the bitstream into 
the HP and the LP partitions. FIG. 4 is a block diagram 

35 of an alternative embodiment which although not main- 
taining strict MPEG compliance, in order to reduce over- 
head and improve performance, can still be used with 
standards compliant MPEG encoders and decoders. 
This embodiment may applied to MPEG-1 or MPEG-2 

40 video, as well as other similar video coding standards 
such as H.261 and H.263. In this embodiment a stand- 
ard MPEG encoder (or other standard encoder) 401 
compression codes the input video stream. A data split- 
ter 402 then divides the compression-coded output of 

45 encoder 401 into HP and LP partitions. For purposes of 
the present invention, the combination of.the standard 
MPEG encoder 401 with a data splitter 402 enables the 
splilling function to be performed more efficiently and 
with improved performance as compared to the data 

50 partitioning MPEG encoder 102 in FIG. 1. Specifically, 
only slice headers are duplicated in both partitions and 
all other data is placed in one partition or the other, but 
not both. Because of the frame alignment of packets and 
the containment of frame information in the packet 

55 headers themselves, it is not necessary to duplicate the 
picture headers and above in each of the partitions, 
thereby minimizing overhead. By providing slice head- 
ers in both the LP and HP partitions, however, a point 
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of entry is provided (or the LP data upon a packet loss. 
Thus, in a received packet that follows a lost packet, the 
LP data can bo incorporated into the decoded picture 
after the decoder receives, with the LP partition data, 
the next shce header Advantageously, as compared the 
embodiment of FIG 1 , in which the point of insertion for 
the received LP data following a packet loss was the 
next frame the embodiment of FIG. 4 provides a point 
of insertion for the LP data at the next slice, thereby im- 
proving the picture quality of the decoded video signal. 
[0031] As previously described, the data partitioning 
MPEG encoder 102 partitions I frames so that all frame 
data is placed in the HP partition to minimize the effect 
of an error in an I frame from propagating to other 
frames. Since B frames are not used for prediction, as 
much data as is possible via the data partitioning stand- 
ard is placed in the LP partition In the embodiment of 
FIG. A, all data for all B frames is placed in the LP par- 
tition as opposed to the embodiment of FIG. 1 in which 
some data, in accordance with the standards, is re- 
quired in the HP partition In both the embodiment of 
FIG. 1 and FIG 4. the P frame data is divided between 
the HP and LP partitions in the manner described above. 
Within each P frame macroblocks can be either inter- 
framc or intra-framc coded. In the embodiment of FIG. 
4, different priohty breakpoints are chosen for inter and 
intra coded macroblocks in each P frame, which the data 
partitioning fV/IPEG encoder 102 in FIG. 1 could not do. 
inter-coded macroblocks decoded using only the HP 
partition (and not the LP partition) may retain high fre- 
quency information from the corresponding motion- 
compensated macroblocks in the previous frame, while 
intra-coded macroblocks may not. Hence, it is desirable 
to set the phority breakpoint for intra-coded macrob- 
locks to include more DOT coefficients in the HP parti- 
tion than are included in inter-coded macroblocks. Ad- 
vantageously, this reduces the overhead rate for given 
level of quality or improves the quality for the same over- 
head rate. 

[0032] The HP and LP outputs of data splitter 402 are 
applied to a FEC coder/packettzer 403= which functions 
in the same manner as FEC coder/packetizer does in 
FIG. 1 , which functions have been described above. The 
packets outputed by FEC coder/packetizer 403 are then 
transmitted over a UDP/IP network. 
[0033] Although the embodiment in FIG. 4 includes a 
separate standard MPEG encoder 401 and data splitter 
402. it should be recognized that a data coder that sup- 
ports the described data splitting operation could equal- 
ly perform the same functions that encoder 401 and 
splitter 402 together do. 

[0034] The encoder networks in FIG. 1 and FIG. 4 
transmit output packets over a UDP/IP network such as 
the public Internet, to corresponding decoders connect- 
ed to that network. In the case of a multicasting encoder, 
the transmission may be simultaneously broadcast to a 
plurality of end-users who will each receive the trans- 
mitted information Since different packets may be lost 
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along the routes to different of such end-users, the FEC 
techniques employed in the present invention to protect 
against packet loss enable each individual decoder to 
recover the particular packets that may have been lost 
5 on the path to that end-user up to. of course, the level 
of packet protection provided by the specific RS code 
used by the encoder. 

[0035] A decoder network 500 that is associated with 
the encoder network of FIG. 1 is illustrated in FIG. 5. As 

10 a data partitioning MPEG encoder 102 is incorporated 
within the encoder network of FIG. 1 , the decoder net- 
work of FIG. 5 incorporates a complimentary data par- 
titioning MPEG decoder 510. In FIG. 5, the serial pack- 
ets transmitted by the encoder network over the UDP/ 

IS IP network 501 are inputted to a de-packet izer/decoder 
502. De-packetizer/decoder 502 includes a de-pack- 
etizer 503 which receives the serial packets and strips 
the header information in each packet and passes it to 
a processor 504. The packet header information in- 

20 eludes; a packet number; a frame number, the type of 
frame (I, B, P); the (n.k) parameters which define the 
packet structure of the frame; and the number of HP par- 
tition bytes, HPS, and LP partition bytes, LPB, in each 
packet. Processor 504, from the header information, de- 

25 tormines the start of the frame, and from the parameter 
a "knows" that that many packets are used to define the 
frame. Further, from the packet numbers received, proc- 
essor 504 determines which particular of those n pack- 
ets, if any. are missing and their position within the se- 

30 quence of these n packets. In response to receiving all 
such information from processor 504, de-packetizer 503 
strips off the packet header of each packet within the 
frame and divides the data in each packet into its HP 
and LP portions. For those packets which are deter- 

35 mined to be missing, de-packetizer 503 inserts "0" bytes 
or an error code in the HP and LP data streams. The HP 
serial byte stream output of de-packetizer 503 consists 
of n sub-packets-worth of HP data, each sub-packet 
containing HPB bytes. This HP stream is inputted to an 

40 interleaver 505 to decode the f\S(n,k) encoded words 
that exist across the sub-packets and to replace any 
missing data from up to h lost packets. Thus, for each 
byte position across the sub-packet, a byte is selected 
from each such sub-packet at that byte position to form 

45 an input word to an RS decoder 506. 

[0036] As an example, using the frame structure 
shown in FIG. 2 where n is equal to 4 and k is equal to 
3. and with HPB being equal 650 bytes, interleaver 505 
selects the 1^, 651«*, 1301®* and 1951^* bytes in the HP 

50 byte stream to form a 4-byte input word to RS decoder 
506 to determine the 3-byte corresponding output word, 
each byte in the three-byte output word being an HP in- 
formation byte in the first byte position in each of the 
three information sub-packets. Interleaver 505 then 

55 presents the 2"^, 652"<*. 1 302^^*^ and 1 952"^ bytes in the 
HP byte stream to RS decoder 506 to determine the 
bytes in the second byte position in each of the three 
information sub-packets. Interleaver 505 similarly proc- 
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esses the 3^^ through the 650*^ bytes. The three infor- 
mation bytes for each byte position are thus sequentially 
outputted by RS decoder 506. Since the RS{4.3) code 
is capable of correcting up to one erasure, if one of the 
four packets is lost and thus not received by the decoder 
network 500. RS decoder 506 will determine the missing 
byte in each of the byte positions in the lost HP sub- 
packet. Thus. If processor 504 determines that the third 
packet is lost. "0" bytes are inserted in each byte position 
from the 1 301 through the 1 950*^ bytes in the HP byte 
stream. Processor 504 passes the location of these 
missing bytes to RS decoder 506, which when sequen- 
tially presented with the bytes from the three received 
packets recovers each missing byte in each byte loca- 
tion in the third packet. Thus, RS decoder 504 is able to 
recover the entire sub-packet of HP data in the single 
lost third packet. II more than one packet is lost in this 
example, or more than h packets in the general case, 
then RS decoder is unable to recreate the missing data 
and a sequence error code that can be recognized by 
the data partitioning MPEG decoder 510 is inserted in 
the HP byte stream in place of the recovered data. 
[0037] Since RS decoder 506 outputs a /c-byte word 
for each n-byte input word, in which each of the /c bytes 
is associated with a different sub-packet, dc-intorloavcr 
507 re-sequences each /c-byte output word, one byte at 
a time, into k separate HP sub-packets to reformulate 
the HP information in each transmitted packet. The HP 
information in these k sub-packets that is re-created 
and. if necessary recovered, is outputted by de-inter- 
leaver 507 and inputted on a first input of the data par- 
titioning MPEG decoder 51 0. Where lost packets are un- 
recoverable, the data stream contains sequence error 
codes. 

[0038] The LP information in the nLP sub-packets de- 
rived by de-packetizer 503 is simultaneously inputted on 
a second input of data partitioning MPEG decoder 510. 
For those packets that are lost, the LP data cannot be 
recovered since forward error/erasure correction was 
not applied to the LP partition. For those packets, there- 
fore, the LP output of de-packettzer 503 includes a code- 
word that will be recognized by the data partitioning 
MPEG decoder 510 as being indicative of missing data. 
[0039] The LP and HP data stream inputs to data par- 
titioning MPEG decoder 510 are equivalent, in the ab- 
sence of any lost packets, to the HP and LP data stream 
outputs of the data partitioning MPEG encoder 102 in 
FIG. 1 . If up to and including h packets are lost, the HP 
data stream inputted to decoder 51 0 is equivalent to the 
HP data stream outputted by encoder 102 and the LP 
data stream inputted to decoder 510 includes code- 
words marking lost data. When more than /7 packets are 
lost so that RS decoder 506 cannot recover the lost HP 
data, both the HP and LP data stream inputted to data 
partitioning MPEG decoder 510 have missing data, the 
HP data stream including sequence error codes indicat- 
ing the absence of actual data and the LP data stream 
including the recognizable codewords indicating lost da- 



ta. 

[0040] Data partitioning MPEG decoder 510, in re- 
sponse to the inputted LP partition data and the HP par- 
tition data decompresses and reformulates the transmit- 

5 ted video data in accordance with its standardized algo- 
rithm. Where, with respect to specific pels in the frame, 
corresponding HP data is available (by being actually 
received or recovered) but LP data is not available, the 
subjective video quality of the reconstructed video frame 

10 is degraded. Furthermore, that spatial portion of the re- 
constructed video frame that follows, in a scanning 
sense, those pels in the frame associated with the lost 
LP partition data, is also degraded since the LP partition 
data in the next received packet cannot be associated 

15 with specific spatial points within the frame and thus with 
the HP data. As noted, only picture headers are included 
within both the HP and LP partitions. Thus, until the next 
picture header is received, all LP partition data within 
the video frame that follows a lost packet will be unable 

20 to be combined with spatially corresponding HP data to 
decode and reconstruct the video signal. Further, since 
the type of frames in this embodiment which are divided 
into separate HP and LP partitions are the P frames, 
which are used to predict the next frame, the loss of LP 

25 data for reconstructing the remainder of the frame will 
have an effect on the quality of the next P and B frames, 
albeit at a substantially reduced level as compared to 
the effect that a total loss of data would have caused 
without the present invention in which the more impor- 

30 tant HP partition data has been protected. In the event 
that more than h packets are lost in transmission, both 
HP and LP partition data is lost and not recoverable. 
Standard error concealment techniques can then be 
used to minimize the degradation in video quality 

35 [0041] A decoder network 600 which is associated 
with the encoder network of FIG. 4 is illustrated in FIG. 
6. In this embodiment of a decoder network, a depack- 
etizer/decoder 601 receives from the UDP/IP network 
the data stream that was transmitted by encoder net- 

40 work of FIG. 4. De-packetizer/decoder 601 functions in 
a manner identical to that of de-packetizer/decoder 502 
in FIG. 5 and includes the same elements as those de- 
scribed in connection with the description above of FIG. 
5. The de-packetizer. interleaver, RS decoder, de-inter- 

45 leaver and processor will not. therefore, be discussed 
in detail. The outputs of de-packetizer/decoder 601 are 
the LP and HP partition data streams, as are the outputs 
of de-packetizer/decoder 502 in FIG. 5. In the absence 
of any lost packets in a frame, the LP and HP partitions 

50 are equivalent to the HP and LP outputs of data splitter 
402 in the encoder network of FIG. 4. If one through h 
packets are lost, then any lost HP data is recovered and 
the HP data stream is equivalent to the output of data 
splitter 402. The LP stream outputted by depacketizer/ 

55 decoder 601 includes, however, a codeword indicating 
that LP data has been tost. Where more than /7 packets 
are lost, then both the HP and LP data streams include 
error codes. Alternatively, the de-packetizer/decoder 
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601 could send information to a data merger 602 indi- 
cating the positions of missing HP and LP information. 
[0042] The LP and HP data streams outputted by de- 
packetizer/decoder 601 are inputted to the data mergsr 
602, which combines these data streams into a single 
data stream having a format which is decodable by a 
standard MPEG decoder 603 MPEG decoder 603, us- 
ing its standard algorithm, decompresses the coded vid- 
eo signal to reconstruct the digital video signal, which 
can then be transformed to an analog video signal and 
displayed on a video terminal. 

[0043] As earlier noted in connection with the descrip- 
tion of the encoder in FIG. 4, slice headers are included 
in both the LP and HP partitions outputted by data split- 
ter 402. Thus, the for decoder in FIG. 6, upon a loss of 
a packet and its LP data, a point of entry can be located 
for the LP data received in subsequent packets upon 
detection of the next slice header in such data. Thus, 
unlike the embodiment of the decoder in FIG. 5 in which, 
as noted, received LP data could only be reincorporated 
with HP data upon receipt of the next picture header for 
the next frame, the embodiment of FIG. 6 enables the 
LP data received in those packets following a lost packet 
to be incorporated with received or recovered HP data 
at a point of entry at tho next slice hoador. Thoroforo. 
only that portion of the frame that is associated with the 
lost LP data until the next received slice header will be 
degraded, rather than the entire remainder of the frame 
as in the embodiment of FIG. 5. Further, by minimizing 
the visually degraded portion of the decoded frame, the 
degradation of the subsequent frames that may be pre- 
dicted based on that frame is also minimized. 
[0044] The preceding merely illustrates the principles 
of the Invention. It will thus be appreciated that those 
skilled in the art will be able to devise various arrange- 
ments which, although not explicitly described or shown 
herein, embody the principles of the invention and are 
included within its spirit and scope. Furthermore, all ex- 
amples and conditional language that have been recited 
hereinabove are principally intended expressly to be on- 
ly for pedagogical purposes to aid the reader in under- 
standing the principles of the invention and the concepts 
contributed by the inventor to furthering the art, and are 
to be construed as being without limitation to such spe- 
cifically recited examples and conditions. Moreover, all 
statements that made hereinabove reciting principles, 
aspectS: and embodiments of the invention, as well as 
specific examples thereof, are intended to encompass 
both structural and functional equivalents thereof. Addi- 
tionally, it is intended that such equivalents include both 
currently known equivalents as well as equivalents de- 
veloped in the future, i.e., any elements developed that 
perform the same function, regardless of structure. 
[0045] Thus, for example, it will be appreciated by 
those skilled in the art that the block diagrams herein 
represent conceptual views of illustrative circuitry em- 
bodying the principles of the invention. Similarly, it will 
be appreciated that any flow charts, flow diagrams, and 



the like represent various processes which may be sub- 
stantially represented in computer readable medium 
and so executed by a computer or processor, whether 
or not such computer or processor is explicitly shown. 

5 [0046] The functions of the various elements shown 
in the FIGS described above, including functional 
blocks labeled as "processors" may be provided through 
the use of dedicated hardware as well as hardware ca- 
pable of executing software in association with appro- 
priate software. When provided by a processor, the 
functions may be provided by a single dedicated proc- 
essor, by a single shared processor, or by a plurality of 
individual processors, some of which may be shared. 
Moreover, explicit use of the term "processor" or 'con- 

15 troller" should not be construed to refer exclusively to 
hardware capable of executing software, and may Im- 
plicitly include, without limitation, digital signal proces- 
sor (DSP) hardware, read-only memory (ROM) for stor- 
ing software, random access memory (RAM), and non- 
20 volatile storage. Other hardware, conventional and/or 
custom, may also be included. Their function may be 
carried out through the operation of program logic, 
through dedicated logic, through the interaction of pro- 
gram control and dedicated logic, or even manually, the 

25 particular technique being selectable by the implement- 
er as more specifically understood from the context. 
[0047] In the claims hereof any element expressed as 
a means for performing a specified function is intended 
to encompass any way of performing that function in- 

30 eluding, for example, a) a combination of circuit ele- 
ments which performs that function or b) software in any 
form, including, therefore, firmware, microcode or the 
like, combined with appropriate circuitry for executing 
that software to perform the function. The invention as 

55 defined by such claims resides in the fact that the func- 
tionalities provided by the various recited means are 
combined and brought together in the manner which the 
claims call for. Applicant thus regards any means which 
can provide those functionalities as equivalent to those 

40 shown herein. 



Claims 

45 1. A method Of encoding a video signal for transmis- 
sion over a packet-based network comprising the 
steps of: 

compression coding the video signal; 
50 splitting at least one frame of the compression- 

coded video signal into high priority partition in- 
formation bytes and low priority partition infor- 
mation bytes; 

applying a forward error/erasure correction 
(FEC) code only to the high priority partition in- 
formation bytes to form FEC-coded high priority 
partition information bytes; and 
arranging the FEC-coded high priority partition 
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information bytes and the low prbrity partition 
information bytes associated with the at least 
one frame into a plurality of packets. 

2. The method of claim 1 wherein the packet-based 
network is the Internet. 

3. The method of claim 1 or claim 2 wherein the for- 
ward error/erasure correction code is a systematic 
forward error/erasure correction code, the FEC- 
coded high priority partition Information bytes com- 
prising a combination of the high priority partition 
information bytes and associated parity bytes. 

4. The method of claim 3 wherein each packet has an 
equal length and each packet contains both high 
priority partition data bytes and low priority partition 
information bytes, the high priority partition data 
bytes in each packet comprising either all high pri- 
ority partilion information bytes or all parity bytes, 
the method further comprising the step of forming 
the parity bytes in one or more of the packets con- 
taining parity bytes by applying the systematic for- 
ward error/erasure code to the high priority partition 
information bytes in the other packets. 

5. The method of claim 4 wherein an equal number of 
low priority partition information bytes are in each 
packet and an equal number of high priority partition 
data bytes are in each packet 

6. The method of claim 5 wherein, for each high prior- 
ity byte position, the forward error/erasure code is 
applied using one high priority partition byte from 
the same byte position from each packet containing 
high priority partition information bytes to determine 
the associated parity bytes for that byte position, the 
associated parity bytes being arranged in that same 
byte position, one byte per packet, in the packets 
containing parity bytes. 

7. A method of encoding a video signal for transmis- 
sion over a packet-based network comprising the 
steps of 

compression coding the video signal; 
splitting at least one frame of the compression- 
coded video signal into high priority partilion in- 
formation bytes and low priority partition infor- 
mation bytes; 

applying a systematic forward error/erasure 
correction code only to the high priority partition 
information bytes to produce an output com- 
prising a combination of the high priority parti- 
tion information bytes and associated parity 
bytes; and 

arranging the low priority partition information 
bytes and the output high priority partition infor- 



nnation bytes and associated parity bytes into a 
plurality of packets, each packet containing low 
priority partition information bytes and eitherall 
high priority partition information bytes or all 

5 parity bytes, for each high priority byte position, 

the forward error/erasure code being applied 
using one high priority partition byte from the 
same byte position from each packet contain- 
ing high priority partition information bytes to 

10 determine the associated parity bytes for that 

byte position, the determined associated parity 
bytes being arranged in that same byte posi- 
tion, one byte per packet, in the packets con- 
taining parity bytes. 

15 

8. The method of claim 7 wherein each packet has an 
equal length and an equal number of low priority 
partition information bytes are in each packet and 
an equal number of high priority partition informa- 

20 lion bytes or parity byles are in each packet. 

9. The method of claim 8 or claim 6 wherein the for- 
ward error/erasure correction code is a Reed Solo- 
mon code. 

25 

10. The method of claim 8 or claim 6 further comprising 
the steps of 

determining for the at least one frame of the 

30 compression-coded video signal the number of 

information bytes in the high priority partition 
and in the low priority partition: 
for a predetermined desired minimum level of 
protection against the loss of one or more pack- 

35 ets, determining for the at least one franne from 

the number of high priority partition information 
bytes, the number of low priority partition infor- 
mation bytes, and a predetermined maximum 
number of bytes per packet: (1) the number 

40 bytes per packet; (2) the total number of pack- 

ets (n) needed for the high priority partition in- 
formation, associated parity bytes, and the low 
priority partition information bytes; and (3) of 
the n packets, the number of packets (k) which 

45 include high priority partition information bytes. 

11. The method of claim 10 wherein forward error/eras- 
ure code is a ReedSolomon (n,k) code. 

50 12. The method of claim 10 wherein the high priority 
partition information bytes of the compress ion -cod- 
ed at least one video frame of the video signal are 
protected against packet loss over the packet- 
based network of up lo (n-k) packets. 

55 

13. The method of any of the preceding claims wherein 
the step of compression coding the video signal us- 
es MPEG coding. 
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14. The method of claim 13 wherein the at least one 
frame is a single 5 frame and the step of splitting 
comprises the step of splitting information bytes of 
the compression-coded video signal into high prior- 
ity partition information bytes and low priority parti- 5 
tton information bytes at a priority breakpoint in 
each macroblock of the compression-coded video 
signal that is determined as a function of whether 
the video frame is an intra-frame coded I frame, a 
predictive P frame, or a predictive B frame. io 

15. The method of claim 14 wherein if the frame is an I 
frame, the priority breakpoint is chosen so that sub- 
stantially all the information bytes of the compres- 
sion-coded video signal are high priority partition 75 
bytes. 

16. The method of claim 14 wherein if the frame is a B 
frame, the priority breakpoint is chosen so that sub- 
stantially all the information bytes of the 15 com- 20 
pression -coded video signal are low priority parti- 
tion bytes 

17. The method of claim 14 wherein if the frame is a P 
frame, data bytes associated with a macroblock 2S 
within the compression-coded video signal are split 

as being high priority partition bytes or low priority 
partition bytes at a priority breakpoint that is deter- 
mined as a function of whether the macroblock is 
an intra-coded or an inter-20 coded macroblock 30 

18. An encoder for encoding a video signal for trans- 
mission over a packet-based network comprising 
means arranged to carry out each step of a method 

as claimed in any of the preceding claims. 35 
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